Gradients Can Be Easier Than You Think!